Skip to content

gh-88037 - Move port validation logic to parsing time - #159122

Merged
orsenthil merged 2 commits into
python:mainfrom
orsenthil:gh-88037-port-validation
Nov 8, 2026
Merged

orsenthil merged 2 commits into
python:mainfrom
orsenthil:gh-88037-port-validation

Conversation

@orsenthil

@orsenthil orsenthil commented Oct 10, 2026 •

Copy link
Copy Markdown
Member

Currently urlsplit()/urlparse() accept a netloc with an invalid port without complaint, the error only surfaces later, and only if something actually reads .port:


    >>> p = urllib.parse.urlsplit("http://example.com:oracle/")
    >>> p.netloc          # parses "successfully"
    'example.com:oracle'
    >>> p.path
    '/'
    >>> p.port             # ... raises, but only now, and only if you ask
    ValueError: Port could not be cast to integer value as 'oracle'

This moves that validation to parse time, so urlsplit()/urlparse() raise immediately on a malformed port instead of returning a result that looks valid until .port happens to be accessed. This in conformance with the RFC expectations.

RFC 3986 3.2:

authority = [ userinfo "@" ] host [ ":" port ]
host      = IP-literal / IPv4address / reg-name
reg-name  = *( unreserved / pct-encoded / sub-delims )   ; sub-delims includes ","
port      = *DIGIT

Supersedes #25774 - Rebase of the old branch wasn't suitable: Lib/urllib/parse.py has seen significant changes since 2021 (WHATWG-compliance work, bracketed IPv6 host validation) that the original patch's diff didn't account for. The original commit is kept in-tact.


I was curious on performance of moving this lazy validation to eager, I let the claude run in tiny benchmark.

Benchmark

Measured with timeit (best of 5 runs) on a local ./python build
of this branch (3.16.0a0), comparing the real, unmodified
Lib/urllib/parse.py from main (lazy validation) against this
branch's version (eager, parse-time validation).

_checknetloc in isolation (500k calls each):

netloc lazy (main) eager (this PR) delta
ASCII, no port 54.0 ns 288.4 ns +234.5 ns
ASCII, with port 53.2 ns 468.2 ns +415.0 ns
Bracketed IPv6 53.6 ns 575.6 ns +522.0 ns

Public urlsplit(), which is @functools.lru_cache-wrapped
(128 entries):

scenario lazy (main) eager (this PR) delta
cache hit (same URL reparsed) 124.1 ns 124.2 ns +0.1 ns (+0.08%)
cache miss (unique URL each call) 2626.8 ns 3131.9 ns +505.1 ns (+19.2%)

Takeaways:

  • When a URL has already been parsed once, the cache absorbs the
    difference entirely — effectively zero added cost.
  • For a never-before-seen URL, the added cost is ~500 ns, sub-microsecond
    in absolute terms and negligible next to any I/O the parsed result is
    normally used for (DNS/connect/TLS are milliseconds; this is
    nanoseconds).

Comment thread Lib/urllib/parse.py Resolved
function where ":" should not be treated as port separator.
@orsenthil orsenthil added needs backport to 3.14 bugs and security fixes needs backport to 3.15 bugs and security fixes labels Nov 8, 2026
@orsenthil
orsenthil merged commit 403f6dd into python:main Nov 8, 2026
56 checks passed
@miss-islington-app

Copy link
Copy Markdown

Thanks @orsenthil for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14, 3.15.
🐍🍒⛏🤖

@miss-islington-app

Copy link
Copy Markdown

Sorry, @orsenthil, I could not cleanly backport this to 3.14 due to a conflict.

Please backport manually with cherry_picker, see the devguide for more information.

cherry_picker 403f6dd31a70613d4c49a4c9954aacb33a0637dc 3.14

@bedevere-app

bedevere-app Bot commented Nov 8, 2026

Copy link
Copy Markdown

GH-161619 is a backport of this pull request to the 3.15 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.15 bugs and security fixes label Nov 8, 2026
@orsenthil orsenthil added needs backport to 3.14 bugs and security fixes and removed needs backport to 3.14 bugs and security fixes labels Nov 8, 2026
@miss-islington-app

Copy link
Copy Markdown

Thanks @orsenthil for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

@miss-islington-app

Copy link
Copy Markdown

Sorry, @orsenthil, I could not cleanly backport this to 3.14 due to a conflict.

Please backport manually with cherry_picker, see the devguide for more information.

cherry_picker 403f6dd31a70613d4c49a4c9954aacb33a0637dc 3.14

@orsenthil

Copy link
Copy Markdown
Member Author

3.14 Backport #161621

@bedevere-app

bedevere-app Bot commented Nov 8, 2026

Copy link
Copy Markdown

GH-161621 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Nov 8, 2026
orsenthil added a commit that referenced this pull request Nov 8, 2026
…) (#161619)

gh-88037 - Move port validation logic to parsing time (GH-159122)

* Move port validation logic to parsing time

* Fix CI. Handle the special case of file://C:\\  parsed by url2pathname
function where ":" should not be treated as port separator.

---------
(cherry picked from commit 403f6dd)

Co-authored-by: Senthil Kumaran <senthil@python.org>
Co-authored-by: Miguel Mendes <miguel.mdebrito@gmail.com>
orsenthil added a commit that referenced this pull request Nov 8, 2026
#161621)

gh-88037 - Move port validation logic to parsing time (#159122)

* Move port validation logic to parsing time

* Fix CI. Handle the special case of file://C:\\  parsed by url2pathname
function where ":" should not be treated as port separator.

---------


(cherry picked from commit 403f6dd)

Co-authored-by: Miguel Mendes <miguel.mdebrito@gmail.com>
orsenthil pushed a commit that referenced this pull request Nov 14, 2026
…161988)

Revert "[3.14] gh-88037 - Move port validation logic to parsing time (#159122) (#161621)"
This reverts commit bc35f90.
orsenthil pushed a commit that referenced this pull request Nov 14, 2026
…161989)

Revert "[3.15] gh-88037 - Move port validation logic to parsing time (GH-159122) (#161619)"
This reverts commit 1a530ac.
@orsenthil

Copy link
Copy Markdown
Member Author

This change was made only to 3.16 - Even as this is a bug report, since there is a behavior difference in parsing, not porting it to older versions to prevent surprises.

I have raised a pip issue to warn about this upcoming change pypa/pip#14438 in Python3.16

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants